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(57) Abstract: A method of communication between a client and a continuous media sender in a data communication backbone 
network, the method comprising the steps of the server composing data lo be uansmitted into a backbone common format; the server 
transmitting the backbone common format data to the client POP; convening at the POP the backbone common format data into a 
plurality of access common format data for transmission to ones of a plurality of clients. Preferably, wavelet compression is used to 
convert data to optimally select a predetermined quality. Accordingly, the present invention solves the problem of network congestion 
by providing a single high quality stream of data to an edge server which may be later filtered to accommodate a client or consumers 
processing capability. 
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System and Method for Streaming Media over an Internet Protocol System 

The present invention relates to the field of network systems, and more particularly 
to a system and method for optimizing the bandwidth of Internet systems. 

5 . 

BACKGROUND OF THE INVENTION 

The Internet today is defined as the interconnection of Internet protocol (IP) - 
based networks. The Internet protocol stack diagram is shown in figure 1 and represented 
in terms of the ISO - 7-layer model, and described in Table I. 

0 

Table L 



Layer 


FuDCtional Description 


Application 


Provide encryption, compression of streaming media, compression of 

images, text compression for e-mail messaging. Negotiates policy and 
access policy trom users to networK resources ana services. 


Presentation 




Session 


Monitors the Quality of Service (QoS) functionality in the network, and 

takes care of congestion control and routing decisions. Offers traffic 
policing, traffic bandwidth management. 


Network 


TCPAJDP Layer. TCP provides connection -oriented services, in which 
there are acknowledgement packets sent between the source and 
destination, and UDP provides connection-less services, in which packets 
are sent between source and destination without acknowledgements being 
returned. TCP and UDP accept and provide the higher level datagrams to 
the session layer, and segment the datagrams into smaller IP packets 
suitable for transfer and receipt fi-om the network. Address management 
services. 


Transport 


TCP/IP 


Data Link 


The Internet Protocol (IP) provides acknowledgements on a hop by hop 
basis. Normally deals with bytes and packets. 


Physical 


Interface to the physical medium at the level of bits, translates packets to 
bits and vice- versa. 
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Various equipment types and products may be associated with the layer 
functionality that they service, as described in Table II. 
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AuthenUcation and Accounting) servers 



^'^"^ '^•'n^roj products, multicast protocols andlii^rilhiiir 
H.323 standards 



related to 



Network 
Transport 



DHCP Servers, gateways 
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Data Link 



Bridges and switches 



Physical 



XDSL modems, analog .ii^ems. cable modems.- w.reless standards from " 
ETSI and other bodies, repeatere 
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deliver this media to Internet end users will yield a congestion problem due to the nature 
of the web. One of the features that has made the web such a success is the ability of one 
user to access another user's information, regardless of where that information is stored, 
what type of computer it is stored on, or what kind of application was used to create it. 
5 Unfortunately, the same flexibility and ease of use features result in a serious contention 
issue, since everyone competes with everyone else for available network resources. 
Streaming technologies for live audio and video over the web have exacerbated this 
problem even further. 

Normally, hyperlinks point to multimedia files that are downloaded in their 

10 entirety to a user's local disk. However, streaming media allows users to watch live video 
and audio as the file is downloading. 

Some of the reasons for these bandwidth restrictions or bottlenecks are described 
as follows. Since the Internet is IP based, all packets must be evaluated by routers to 
determine the destination delivery paths, creating traffic congestion, particularly with the 

IS increased demand in realtime media, such as video and audio. It has been found that 

backbone, subnet and router upgrades are not sufficient to increase the Internet throughput 
to offset the increasing bandwidth requirements of the WWW itself. This problem is 
further exacerbated by end users having faster access to the ISP POP (Internet Service 
Provider Point of Presence). Simply providing **bigger pipes" to the POP simply sends 

20 bigger chunks of data onto the web. Realtime protocols and specialized backbones have 
been developed. However, these solutions are suitable only for improving transports for 
scheduled or premium events, but are unsuitable for the proliferation of multimedia 
content that is expected in the near ftiture. Although improved compression techniques 
promise to squeeze multimedia files into smaller and smaller sizes, video and audio will 

25 continue to require a "big pipe" as a result of the realtime transport requirements. 

A solution to the above problem has been proposed by Oracle Corporation, In this 
solution, it is proposed that the multimedia data repository is placed closer to the consumer 
of the multimedia. Thus, servers are deployed at the edge of the WWW and multimedia 
data is replicated on these edge servers where the users connection terminates at the POP. 

30 Hyperlinks on the web pages become pointers to streaming media servers that are 

physically closest to the consumer. The philosophy behind this implementation is that the 
POP is the logical termination of the user's access point, and thus packets flowing into or 
out of the POP are only limited by the access speed of the user's connection. Any data 
packets that flow behind or through the ISP back channel, for example, router, are affected 
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by bottlenecks. Thus, by placing the media repository at the POP and behind the router 
the user is insulated from traffic conditions that exist on the Internet at any given time It 
.s envisioned that content providers and u^eb publishers use a combination of mirroring or 
caching techniques to replicate data to the edge servers. 

A disadvantage of the above scheme is that it requires the content providers and 
web publishers to themselves stage propagate and update multimedia data to be replicated 
m the muroring model, whereas, in the caching model, if the requested data by the user 
was not already cached, a dialogue box would info™ the user with the approximate time 
the media would be available and might suggest that they visit other sites in the interim 
In general, this is unacceptable to most user, since most users require instant access to the 
requested data. 

A fiuther improvement on this method, and particularly applicable to streaming 
media, has been proposed by Real Networks which introduced a distributed multitier 
broadcast architecture for the inten,et termed the real broadcast network (RBN). 

In this solution, access to the RBN server is distributed throughout the Internet 
backbone. Live feed is transmitted directly to splitten; which are located in the major 
backbone provider's network. This feed is then retransmitted or "split" from the backbone 
provider to splitters installed at the ISP site, where it is finally streamed to the user's 
computer. 

Another solution that Real Networks proposes to counter the problem of providing 
lugh quality media (video and audio) to streaming users while accommodating the various 
physical connection speeds between the user and the ISP. is to create a scalable stream 
where the server can reduce the amount of data being sent to keep the client from 
rebuffering. This approach is generally referred to as video "stream-thimiing" The 
limitations of this approach is that a video or audio file designed to play at one data rate 
and subsequently scaled down to a lower rate results in an inferior quality level when 
compared to a video optimized specifically for the lower data rates. Furthemiore audio 
codecs cannot usually dynamically send lower data rates. An approach to address this 
heterogeneous comiection rate environment is to create several files so that when a client 
comiects. the server streams the appropriate file. This has been referred to as "bandwidth 
negotiation". This process is not dynamic, so if a user's actual throughput changes due to 
congestion or packet loss, the server camiot adjust. Another difficulty is the increased 
labour required for encoding and then managing the media clip for different bandwidths 
The Real Networks solution to these problems, in its most recent incamation. is to provide 
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an encoding framework for combining multiple data streams, each at different bit rates, 
into a single file. A sophisticated client server mechanism is provided for detecting 
changes in bandwidth and translating those changes into combinations of different 
streams. 

5 While the above attempts to address the solution of bandwidth negotiation 

and stream thinning, it still suffers from the limitation in that multiple streams 
corresponding to different bit rates must be composed at the server end. For 
example, if ten different streams are to be composed at the server to the backbone, 
where each stream ranges from 1 megabit per second as follows: stream 1 - 1 

10 Mbps, stream 2 - 500 Kbps, stream 3 - 300 Kbps, stream 4 - 200 Kbps, stream 5 - 
100 Kbps, stream 6-56 Kbps, stream 7 - 33 Kbps, stream 8- 28 Kbps, stream 9 - 
14 Kbps, stream 10-8 Kbps, then all ten streams must be sent over the backbone 
from the server to splitters and POPs. Thus, a 3.4 Mbps stream is sent down the 
backbone. At the POP, ten different caches are now required. The POP then 

15 forwards the appropriate bit stream to the user depending on the user's access 
capability. It may be seen that in this solution, the user is provided with a relatively 
consistent stream. However, it still does not alleviate the problem of backbone 
congestion since multiple streams must all be transmitted along the backbone, with 
these multiple streams requiring increased bandwidth. 

20 There is thus a need for a system and method for mitigating at least some of the 

above disadvantages. 

SUMMARY OF THE INVENTION 

An advantage of the present invention is to provide a distributed Internet 
25 architecture that minimizes congestion on the Internet backbone. 

In accordance with this invention there is provided a method of communication 
between a client and a continues media server in a data communication backbone network, 
the method comprising the steps of the server composing data to be transmitted into a 
backbone common format; the server transmitting the backbone common format data to 
30 the chent POP; converting at the POP the backbone common format data into a plurality 
of access common format data for transmission to ones of a plurality of clients. 

5 
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In accordance with a further embodiment of the invention the client includes the 
step of filtering the received data according to a predetermined parameter of the quality. 

A still further embodiment of the invention provides for the conversion of the 
backbone common format data to wavelet compressed data. 

BRIEF DESCRIPTION OF THE DRAWINGS 

TTiese and other features of the preferred embodiments of the invention will 
become more apparent in the following detailed description in which reference is made to 
the appended drawings wherein: 

Figure 1 is schematic diagram of protocol stack diagram; 
Figure 2 is schematic diagram of an Internet architecture; 
Figure 3 is a schematic diagram of an Internet architecture according to an 
embodiment of the present invention; 

Figure 4 is a schematic diagram of a multiple consumer connection architecture 
accordmg to an embodiment of the present invention; 

Figure 5 is a schematic diagram of an edge server client connection; 
Figure 6 is a detailed schematic diagram of a edge server; and 

Figure 7 is a schematic diagram of a streaming media server with an edge server 
transcoding. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following description like numeral refer to like stnactures in the drawings 
Referring to figure 2. a general Internet architecture as it currently exists is shown 
generally by numeral 20. Tbe architecture comprises a backbone network 22 which is 
defined as the interconnection equipment concerned with connecting local web sites to 
local POPs and an access network 24 that is defined as the intercomiection between the 
local POPs and the consumers. The Web sites 26 host both digital and analog content 
from various Content providers 28. which are in-turrt comiected via a global and national 
mtemet mfrastructure 30 to the local access internet infrastructure 32. The consumer or 
viewer 34 comiects to the national Internet infrastructure 30 i.e. at the POP by a one or 
more access links 33 as defmed earlier. Cache 36 sites are provided between the global 
and national Internet infrastructure 30 and the local access Internet infrastructure 32 The 
cache sites 36 are nonnally the demarcation between the backbone network 22 and the 
access network 24. Backbone web sites 26 typically do not consider the needs of various 
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types of access 33 employed by clients 34 and various qualities of access links in their 
consideration of web content. It is normally the responsibility of the web content provider 
28 to customize the web site content for different access links. 

Therefore the present invention leverages off the existing architecture, but 
5 implements a content compression architecture that uses data link level 6 to application 
level 16 technologies to optimize the access from the local POP to the customer 34. This 
architecture relieves the web hosting and web content provider from the burden of 
customizing the web site content or protocol for different access links and moves the 
access specific customization to the local POP site 32. This content compression 

10 architecture for Internet IP networks uses a concept of backbone common format to access 
common format (BCS-ACS) Translation. 

Referring to Figure 3 a network architecture for streaming media data according to 
the present invention is shown generally by numeral 60. The architecture 60 includes 
enhanced data compression 62 to existing or new web sites 26; caching algorithms 64 for 

1 5 the connection of these web sites 26 to existing 36 or new 37 local cache sites; local cache 
and serving products with compression translation, local compression and access specific 
translation 66 at the POP; and new hardware and software client technologies for PC and 
embedded PC consumer sites 68. Each of these enhancements will be discussed in detail 
below. 

20 Referring to Figure 4 a schematic diagram of the access network 24 according to 

an embodiment of the present invention is shown generally by numeral 70. As described 
above, the web sites 26 are connected via the backbone 72 to the local access equipment 
72. Thus the web site server composes streaming media data to be transmitted into a single 
stream of backbone common format data. The server transmits this single stream 

25 backbone common format data to the client POP. However, at the local access equipment 
level 74 there are conversions performed that translate the single stream backbone 
common format data into different common access formats that are suitable for optimized 
transmission to consumers 78. The data may be transferred via a wireless 3G link 80, a 
copper DSL link 82, a coaxial cable link 84, or other access links known in the art. These 

30 particular three methods are simply exemplary of three different types of access media. 
The wireless access media 80 is inherently broadcast and simulcast capable, the copper 
media 82 is inherently point to point, and the cable interconnection 84 is inherently bus 
broadcast and simulcast capable. 
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compress it further lo access standard compression methods based on wavelet 
compression. This function will allow the invention to retrofit into the existing Internet 
connections. The transcompression will be of the streaming variety and therefore, the 
technology is fast enough to handle multiple simultaneous streams. The local access site 

S also includes a modified congestion algorithm for the access link. It converts the 
backbone header structure into smaller headers and resizes the IP packet size from 
backbone size to access size. Standard management information bases (MIB) are 
accessible over the network by SNMP. The system can be upgraded using software, with 
a software delivery method such as download available over the web. The system can also 

10 interface to the various access networks using OEM equipment from various vendors. 

Referring to Figure 6 a schematic diagram of the transcompression module shown 
in Figure 5 is shown. Content from the backbone websites are received and passed via an 
HTTP level switching module to a digital CCIR 656 interface. A CCIR 656 interface 
couples the data to an optimized wavelet compressor. The compressed data from the 

15 compressor is sent to a web-site based storage under the control of storage logic. The 
storage logic is also capable of forwarding received data directly to the storage without it 
being compressed. The compressed content is made available to a streaming server for 
eventual serving to a client. 

The choice of wavelet compression offers multiple advantages over other 

20 compression schemes. At the local site, wavelet technology can be used in the 

consideration of the speed of the access pipe. Router technology, gateway technology and 
other Internet infrastructure equipment may benefit from allowing more or less wavelet 
coefficients through the channel. The number of wavelet coefficients used determines the 
quality at which the compressed data is uncompressed. This functionality is provided by 

25 allowing the wavelet compression stream to have priority tags on the packets which 
indicate those coefficients that are most important for the data stream and those 
coefficients that are less important. Wavelet compression is good for this particular 
function because the primary, secondary and tertiary coefficients are clearly definable. 
This aspect of the wavelet technology is most useful for point to point connections such as 

30 copper DSL links. If the consumer is connected to the Internet via a slow connection, 
such as a 14k modem, the local equipment can send the compressed data with fewer 
coefficients. While this may reduce the quality of the data received by the consumer, it 
will allow the consumer to view the data at perceptively improved rates. Although the 
actual rate of transmission does not change, since there is less data to be sent the customer 
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perceives the transmission to be faster. If the customer has a faster connection to the 
Internet, then the local client will send more coefficients and the client will be able to 
reccve better quality data. Therefore, this method improves the speed of transfer of data 
by reducing the amount of data to be sent. 

Another benefit of the wavelet technology is that for links with multiple clients 
accessing the same data stream, only one fonn of the data needs to be sent. Wavelet 
compression allows for progressive video. Progressive video occurs when a single 
compressed bitstream. using wavelets, is sem to various forms of destination capability 
Since the data is non-interlaced, the data can be uncompressed linearly with different 
wavelet coefficients depending on the processing power of the client. If. for example a 
single wavelet stream is sent to a mobile user with a Palm Pilot, a PC using a slow 
processor, and a PC using a fast processor, the stream would be processed differently in 
eachoftheseenviromnents. For the Palm Pilot™ environment, the screen is small and 
the processing speed is small and therefore, only small portions of the wavelet coefficients 
are used to create the picture. For the slow PC. the screen resolution is greater and more 
wavelet coefficients are used. For the faster PC. the screen resolution is the highest and 
the processing speed allows all ofthe wavelet coefficients to be used. Therefore the 
encoding source does not need to be concerned about the capability ofthe end device and 
mstead, compresses at maximum capability and allows the end device to use whatever 
portion ofthe wavelet coefficient i, needs. This method does, however, require the dient 
to have the appropriate hardware and software to decompress the wavelet stream. 

Other forms of compression may also be used, such as fractal, vector quantization 
or wavelet-fractal compression as known in the art. 

Local cache is used when fast response times are required for large content 
requests. TTie use of local cache assumes that some web sites and content sites are more 
popular than others, and therefore. local storage of popular content wil, allow for a faster 
response time and decreased backbone bandwidth utili^atio^. TTie extreme end ofthe local 
cache would be if all the coment on the Interne, were sent to each local cache all the time 
In this case, the local cache memory size would need to be as large as the rest ofthe 
30 Internet combined, a scenario that is price prohibitive. 

Referring to Figure 7. a further embodiment ofthe invention is shown generally by 
numeral 700. In this embodiment, the web server at a web site 726 lakes the content 728 
and applies wavelet compression 730 to the content. Thus, the web site 726. provides 
streaming media over an IP that is compliant with a wavelet compression. In this 

10 
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embodiment, the web server 726 responds to media requests received over an IP with 
instructions to compress analog data that is compliant with CCIR recommendation [601 or 
656]. The web server 726 will then send wavelet compressed media data that is compliant 
with the wavelet codec al the POP 732, either in multicast streaming format, unicast 
5 streaming format or the file transfer protocol (FTP) format at a scheduled time. The 
wavelet media server 731 is network manageable with SNMP commands and supports 
standardized MIB (Management interface base) data. 

The wavelet video compression fbnction 730 supports up to 30 fps, both realtime 
wavelet video compression and non realtime wavelet video compression. The wavelet 

10 video compression function 730 receives CCIR comphant analog or digital data from the 
analog video source 728 and sends wavelet compressed video data to the media database 
731. The wavelet video compression function adds a timestamp data to each compressed 
frame as defined by RTP to support a client audio video lip-synch. The video 
compression function receives from the client 734 video compression control information 

15 to start wavelet compression for IP streaming. The wavelet algorithm compresses 30 fps 
QCIF/CIF video al bit rates ranging from 50 Kbps to 1 Mbps depending on the subjective 
video quality to be streamed over the backbone 736. 

On the other hand, analog audio data will be compressed in realtime based on the 
audio compression control information received from the client. The audio compression 

20 function also adds a timestamp data per compressed audio packet compliant with RTP to 
support the client audio video lip-synch algorithm. The MMIP audio compression 
function sends compressed audio data to the media database 73 1 for efficient streaming 
over the backbone network. 

The media streaming function 726 sends a wavelet compressed media data in a 

25 backbone optimized compressed resolution with the media timestamps compliant with 
RTP based on the media streaming control information. The media streaming function 
also supports the sending of data in multicast streaming format , unicast streaming format 
or FTP at a scheduled time. The media streaming function also supports media multicast 
over RTPAJDP to multiple destinations to minimize the loading on the backbone network. 

30 hi addition, support is provided for unicast RTP/TCP for transmissions through firewalls 
for transmissions to destinations which do not support IP multicasting. 

The media streaming function uses an algorithm for coding and timestamping of 
the audio and video packets read from the media database. Since the time difference 
between packets is relatively constant, the second order differences generally approaches 

11 
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zero. An algorithm for a variable key packet (video and audio) is used for lossless 
compression of the time redundancies in the RTP/UDP headers. The algorithm utilizes 
variable length first and second order differences for the exact reconstruction of the 
timestamps by the destination and minimization of backbone network loading. In 
addition, up to 2:1 lossless compression can be achieved for the packet payload if the web 
data is not already in an encrypted or compressed fonnat. An algorithm is used to detect if 
the compression of the web data results in data expansion. Data expansion is typical 
when the compressed algorithm is applied to encrypted data. 

A compression manager function monitors the operational status of the web server 
and uses COTS hardware and software to support design reuse. The compression manager 
receives media requests and reads compressed media data from the media database to 
implement the following functions in maintaining the database, a determination of the 
newly compressed media that is to be maintained in the server; determining the amount of 
each media program to be maintained in the server, setting a time for each media program 
15 to be kept in a server before it becomes stale; congestion algorithms for the backbone 
network for unicast and multicast transmissions to support an anytime/anywhere 
capability; and IP packets segmentation optimizations for the backbone network. 

This optimization shall focus as a minimum on the avoidance of IP segmentation 
and reassembly delays and overhead in the backbone equipment. 

In the above embodiment, it may be seen that compression is moved to the web 
server while transcoding, i.e.. modification of quantities is done at the caching server at 
the ISP or POP. 

Although the invention has been described with reference to certain specific 
embodiments, various modifications thereof will be apparent to those skilled in the art 
25 without departing from the spirit and scope of the invention as outlined in the claims 
appended hereto. 



20 
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THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE 
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS: 

1. A method of communication between a client and a continues media server in a 
5 data communication backbone network, the method comprising the steps of: 

(a) the media server composing data to be transmitted into a backbone common 
fomiat; 

(b) the media server transmitting the backbone common format data to the client 
POP; and 

10 (c) converting at the POP the backbone common format data into a plurality of 

access common format data for transmission to ones of a plurality of clients. 

2. A method as defined in claim 1, including the step of filtering said access format 
data for use by ones of said clients. 

15 

3. A method as defined in claim 2, said filtering being the selection of the data 
compliant with a clients access speed. 

4. A method as defined in claim 2, said selection being performed at the client. 

20 

5. A method as defined in claim 2, said selection being performed at the POP. 

6. A method as defined in claim 1, said converting including applying wavelet 
compression to said backbone common format data to generate said access 

25 common format data. 

7. A method as defined in claim 6, including the step of selecting one or more of said 
wavelet coefficients for transmission to a client. 

30 8. A method as defined in claim 1, said step of converting including applying wavelet 
compression to said backbone common format data. 

9. A method as defined in claim 1, said step of converting including applying fractal 
compression to said backbone common format data. 

13 
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1 0. An edge server comprising: 

(a) a transcompression module for converting a streamed backbone common 
formal data to an access common fomiat data; and 
5 (b) a memory for caching said converted data. 

11. An edge server comprising: 

(a) a transcompression module for receiving data from a network backbone in a 
first format and for converting said received data to a second fomiat; 
^ ^ 0>) a memory for caching said converted daU; 

(c) an interface for coupling said cache to a plurality of clients. 

12. An edge server as defined in claim 1 1 , said second format data including a 
plurality of data streams. 

15 

13. An edge server as defined in claim 1 1 , said second fomiat data being wavelet 
compressed data. 

14. An edge server as defined in claim 12, including a filter for selecting ones of said 
20 plurality of second data in accordance with a predetenmined parameter of a client. 

1 5. A method of communication between a client and a continuos media server in a 

data communication backbone network, the method comprising the steps of: 

(a) the media server compressing data to be transmitted into a backbone common 
25 format; 

Cb) the media sender transmitting the backbone common format data to the client 
POP; 

(c) transcoding at the POP the backbone common format data into a plurality of 
access common format data for transmission to ones of a plurality of clients. 



30 



16. A method as defined in claim 15, said step of compressing including wavelet 
compression. 
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1 7. A method as defined in claim 1 5, said step of transcoding including selecting one 
or more coefficients of said wavelet compressed data for transmission to a client 
and wherein the selection is determined by the clients access speed. 

18. A method as defined in claim 15, said step of transcoding including selecting one 
or more coefficients of said wavelet compressed data for transmission to a client 
and wherein the selection is determined by the clients desired display capabilities. 
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